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PREFACE 


This  Memorandum  Is  part  of  RAND's  continuing  research  into  the 
statistical  analysis  of  computer  simulation  experiments.  Preceding 
work  on  this  subject  has  been  described  in  G.  S.  Fishman  and  P.  J. 
Kiviat,  Spectral  Analysis  of  Time  Series  Generated  by  SimulatLon  Models, 
The  RAND  Corporation,  RM-4393-PR,  February  1965,  and  G.  S.  Fislui.un, 
Problems  in  the  Statistical  Analysis  of  Simulation  Experiments:  The 
Comparison  of  Means  and  the  Length  of  Sample  Records,  The  RAND  Corpo¬ 
ration,  RM-4880-PR,  February  1966.  The  purpose  of  this  research  is 
to  find  methods  for  efficiently  extracting  useful  information  from 
time  series  generated  by  these  experiments.  This  study  compares  two 
simulation  experiments  and  investigates  how  to  efficiently  allocate 
computer  time  between  them.  In  addition,  it  provides  a  two-step  pro¬ 
cedure  for  implementing  the  derived  allocation  rule. 

This  Memorandum  replaces  RM-5288-PR.  The  method  described  here 
leads  to  improved  population  estimators  by  reducing  the  error  due  to 
sampling  fluctuations  and  correcting  for  a  suboptimization  performed 
in  the  earlier  work. 
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SUMMARY 

This  Memorandum  investigates  the  problem  of  efficiently  alloca¬ 
ting  computer  time  between  two  simulation  experiments  when  the  objec¬ 
tive  is  to  make  a  statistical  comparison  of  means.  For  a  given  level 
of  accuracy  our  results  show  that  significantly  less  computer  time  is 
required  when  the  sample  sizes  are  determined  according  to  a  certain 
rule  than  when  the  sample  sizes  are  equal. 

A  graphical  analysis  suggests  that  small  errors  in  estimating 
the  population  parameters  of  the  allocation  rule  do  not  significantly 
affect  the  efficient  allocation  of  time.  The  influence  that  the 
degree  of  autocorrelation  has  on  the  time  allocation  is  alsc  inves¬ 
tigated;  results  show  that  small  differences  in  the  autocorrelation 
functions  are  Important  when  each  process  is  highly  autocorrelated . 

Positively  correlated  samples  for  the  two  experiments  are 
examined  and  incorporated  into  the  efficient  allocation  rule.  It 
is  shown  that  their  use  leads  to  a  saving  in  computer  time. 

A  two-stage  procedure  is  described  wherein  initial  estimates 
of  the  population  parameters  are  computed  which  permit  the  experi¬ 
menter  to  estimate  how  many  more  observations  to  collect  on  each 
experiment.  The  procedure  is  simple  and  straightforward  to  implement 
and  should  be  of  practical  value. 

When  the  computer  time  requirements  turn  out  to  be  prohibitive, 
we  suggest  using  negatively  correlated  replications  on  each  experi¬ 
ment.  This  may  be  accomplished  by  using  antithetic  variates.  The 
two-stage  procedure  also  applies  in  this  case. 
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THE  ALLOCATION  OF  COMPUTER  TIME  IN  COMPARING 
SIMULATION  EXPERIMENTS 


l .  INTRODUCTION 

In  performing  computer  simulation  experiments,  it  is  desirable 
to  minimize  the  computer  time  required  to  obtain  a  statistical  result 
with  a  given  level  of  accuracy.  This  minimization  is  also  desirable 
when  comparing  two  experiments,  but  minimizing  the  time  needed  to 
obtain  a  given  accuracy  for  each  experimental  result  is  not  neces¬ 
sarily  the  same  as  minimizing  the  time  needed  to  obtain  this  same 
accuracy  for  comparing  the  two  results.  It  Is  therefore  instructive 
to  investigate  the  problem:  When  comparing  two  experiments,  how  many 
observations  should  be  collected  from  each  one  to  obtain  a  given  level 
of  statistical  accuracy  with  a  minimum  expenditure  of  computer  time? 

This  problem  Is  investigated  here  with  regard  to  comparing  the 
means  of  two  computer  simulation  experiments.  A  number  of  consider¬ 
ations  enter  into  the  problem's  solution.  The  processes  being  ob¬ 
served  are  generally  autocorrelated ,  thereby  precluding  the  use  of  the 
statistical  methodology  applicable  to  independent  observations.  The 
variances  of  the  sample  means  as  well  as  the  computation  times  re¬ 
quired  to  collect  one  observation  in  each  experiment  strongly  influence 
the  choice  of  efficient  sample  sizes.  Variance-reduction  techniques 
also  play  a  role.  Inducing  positive  correlation  In  the  samples  of  the 
two  experiments  and  negative  correlation  between  replications  of  the 
same  experiment  reduces  the  time  required  to  obtain  a  given  accuracy. 

In  this  Memorandum  a  rule  for  choosing  sample  sizes  Is  described. 
The  rule  incorporates  the  above-mentioned  considerations  and  deter¬ 


mines  the  sample  sizes  necessary  to  meet  a  specified  reliability  with 


1 
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a  minimum  expenditure  of  computer  time.  The  variance  of  the  differ¬ 
ence  of  the  sample  means  measures  reliability. 

A  two-stage  procedure  is  aLso  presented  wherein  Initial  samples 
collected  on  each  experiment  are  used  to  estimate  the  relevant  popu¬ 
lation  parameters,  ftiese  estimates  are  then  used  to  determine  how 
many  more  observations  should  be  collected  to  efficiently  meet  the 
specified  reliability. 


2.  DEFINITIONS 

Consider  a  covariance  stationary  stochastic  sequence  [Xt;  t  «  0, 
±1 ,  ±2 ,  . . . ,  with  mean 


(la) 


*  E(Xt)  , 


variance 


(lb) 


O2  -  E(Xt-vl)2, 


autocorrelation  function 


(lc) 


-2 


Ps  »  c  E[(Xt-li)(Xtfa-n)]> 


(Id) 


l  Im  p  m  0  , 


and  spectrum 


(le) 


g(X)  -  n'Iq2y^pi 


e  0  £  X  <  n. 


S--S0 


Given  a  time  series  of  N  observations,  we  compute  the  sample  mean  as 
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(2a) 


-"“Ev 


t=i 


with  variance 


(2b) 


N-l 

var(30  =  N  1ct2  ^  (l-(s|/N)pg. 
S--NH 


As  N  becomes  large,  we  have 


(2c) 


lim  N  var  (%)  =  m  <  » 
If-® 


(2d) 


m  =  TTg(O)  =  a  p  <- 


so  that  the  large  sample  variance  of  X  is  approximately  m/N.  In 

_  v 

addition  the  statistic  (X-|i) /(m/N)  *  asymptotically  has  the  normal 
distribution  with  zero  mean  and  unit  variance.  We  hereafter  assume 
N  is  large  enough  so  that  these  limiting  properties  hold  reasonably 
well. 

Condition  (Id)  assumes  the  absence  of  perfectly  periodic  compo¬ 
nents  in  {X  ].  If  any  are  present,  they  should  be  removed  prior  to 
performing  the  analysis  presented  here.  Presumably  the  mechanism 
generating  {X^}  will  be  well  enough  understood  to  permit  removal. 

An  analysis  of  the  sample  spectrum  of  (X^}  as  described  in  [4]  will 
reveal  periodicities  if  they  are  present. 

Extremely  narrow  peaks  in  the  spectrum  indicate  the  presence  of 
fairly  regular  periodic  components  in  {X^}.  The  magnitudes  of  these 


See  [ l]  ,  p .  254 . 
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peaks  compared  to  that  of  the  remaining  spectrum  determines  the 
relative  contributions  of  these  components  to  the  mean-square  varia¬ 
tion  in  {Xt}.  In  general  a  spectrum  analysis  can  only  be  performed 
after  the  experiment  is  run.  It  nevertheless  serves  as  a  convenient 

check  on  assumption  (Id)  in  addition  to  providing  other  useful  infor- 
* 

mation. 

Hie  reader  should  not  construe  the  term  "time  series"  to  apply 
only  to  a  set  of  observations  collected  at  equal  intervals  along  a 
time  axis.  The  subscript  t  is  simply  an  index  and  any  series  of 
events  indexed  on  t  can  generate  a  time  series.  For  example,  in  a 
single-server  queueing  problem,  t  may  denote  the  t^  job  to  receive 

irk 

service  and  Xt  may  be  the  waiting  time  of  this  job. 

Our  purpose  is  to  collect  observations  on  a  particular  process 

in  two  simulation  experiments  and  to  compare  the  sample  means  of  the 

two  time  series.  Let  the  subscript  l  denote  experiment  i.  The i  X. 

l » t 

2 

with  mean  ^  ,  variance  and  autocorrelation  function  denotes  the 
ttfl  observation  on  experiment  i. 

Suppose  that  {X.  }  and  {X.  }  are  correlated  with  covariance 

L  ,  C  L.  ,  C 

function 

(3.)  *12,T  -  «t(Xl>t-»,)«2>WTTa)]. 

(3b)  lira  R,-  -  0. 

J-KO  1  *  >T 

Given  Nj  and  observations  on  experiments  l  and  2,  respectively,  we 
have 

See  [4] . 

**For  a  further  discussion  of  the  statistical  analysis  of  series 
of  events,  see  [2]. 
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(4b) 
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3 .  THE  EFFICIENT  ALLOCATION  OF  TIME 

We  denote  the  computer  time  necessary  to  collect  N(  and  N2 
observations  on  experiment  l  and  2,  respectively,  as 


(5a) 


T  =  clNl  +  c2N2 


where  cL  is  the  computer  time  required  to  collect  one  observation  on 
experiment  i.  Suppose  we  require  that 


(5b) 


var(Xl-X2)  ~  V  «  m  /N  +  m  /H 


V  being  specified  by  the  experimenter.  We  choose  and 
minimize  T  for  fixed  V  by  considering 

(5c)  L  =  clNl  +  c2N2  +  x(v-m1/N1-m2/N2) - 

X  being  a  Lagrange  multiplier.  Minimization  yields: 


so  as  to 


(6a) 


N1  =  miCl  +  (br)*1 /V, 


(6b) 


*  *  V 

N2  =  N  j  ( r  / b)  * 


(6c) 


T0  “  (c2m2)^]2/V, 


(6d) 


b  -  c2/Cj  , 


(be)  r  .  m2  /ni|  . 

The  converse  problem  of  minimizing  V  for  a  given  T  has  the  same 


so lu  t  ton . 
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It  is  convenient  to  first  assay  the  special  but  common  case 


Table  l  shows  the  relationship  between  r  and  Tj/Tq  for  this  case. 
For  r  equal  to  8,  roughly  75  percent  of  the  computer  time  is  spent 
on  experiment  2.  When  r  lies  in  the  interval  64  to  128,  approxi¬ 
mately  nine  observations  are  collected  on  experiment  2  for  every 
observation  collected  on  experiment  1. 

Table  1 

RELATIONSHIP  BF”T' .  EEN  r  AND  T*/TQ  =  c2N2/T0  FOR  b  =  1 


r 

T  /T 

2  0 

i 

.5000 

2 

.5853 

4 

I  .6667 

8 

.7387 

16 

.8000 

32 

.8456 

64 

.8889 

128 

.9188 

In  many  simulation  experiments  it  is  customary  to  set 


(7a) 


Nj  =  N2  =  T/(2c) 


so  that  the  computer  time  is  equally  shares.  Then  for  a  given  V, 
have 


(7b) 


T  -  2cm  ( l+r) /V , 


whereas  the  derived  rule  yields 


Tq  -  cm^i+rV/V. 


(7c) 
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To  measure  efficiency  we  use  the  ratio 


(8a)  T/TQ  -  2(l+r) /(1+rS2- 

For  arbitrary  N^  ,  ^  and  b  we  measure  efficiency  by 

(8b)  T/TQ  =  [(l+ab)(l+r/a)]/[l+(br)*]2. 

(8c)  a  =  N2/N1  a  >  0. 

Notice  that: 


(9a) 


l  im  T/T_  =  L  +  ab, 
r— 0  u 


(9b) 


1  Lm  T/T  =  I  +  1  / (ab)  , 

r~-«  u 


(9c) 


lfm  T/T,,  =  l  +  r/a, 
b-0  0 


(9d) 


llm  T/T_  -  l  +  a/r. 
b-a>  0 


Expression  (8b)  suffices  for  comparison  purposes. 

4.  THE  SENSITIVITY  OF  EFFICIENCY  TO  THE  CHOICE  OF  a 

Estimating  c^  and  c^,  and  hence  b,  can  be  accomplished  by 
straightforward  observation.  The  true  value  of  r  is  more  elusive 
and  it  is  therefore  instructive  to  examine  how  sensitive  the  efficiency 
measure  T/T^  is  to  different  values  of  a.  Figure  l  shows  T/T^  for  a 
equal  to  l  ,  2  and  3,  and  b  equal  to  unity.  The  values  of  a  corres¬ 
pond  to  devoting  1/2,  2/3  and  3/4  of  the  computer  time,  respectively, 
to  experiment  2. 

If  6  5  r  S  20,  choosing  a  equal  to  3  yields  an  allocation  close 
to  the  efficient  one.  For  r  <  6  the  allocation  a  equal  to  2  appears 


Fig. 
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reasonably  efficient.  Now  If  r  exceeds  LOO,  the  time  allocation  a 
equal  to  3  requires  45  percent  less  time  to  meet  the  specified  accuracy 
than  an  equal  time  allocation  does.  For  r  of  this  magnitude,  an  a 
equal  to  9  would  be  even  more  reasonable  since 

Urn  T/T  -  10/9,  a  -  9. 
r-eo  u 

Figure  l  shows  that,  while  the  true  value  of  r  may  be  unknown,  a 
rough  estimate  permits  the  experimenter  to  choose  an  a  that  brings 
the  time  allocation  close  to  the  efficient  one. 

Table  2  shows  several  interesting  comparisons.  It  is  evident 
that  as  b  Increases,  the  a  equal  to  3  allocation  becomes  more  desir¬ 
able  for  larger  r. 


Table  2 


T/TQ  FOR  a  -  1,  3  AND  b  =  .25,  .5,  2  AND  4 


b  = 

.25 

b  = 

.5 

b 

=  2 

b 

=  4 

r 

a  =  1 

a  =  3 

a  a  1 

a  =  3 

a  =  1 

a  =  3 

a  =  1 

a  =  3 

i 

l.im 

1.0370 

1.0294 

1.1438 

1.0294 

1.6013 

l.iin 

1.9259 

2 

1.2868 

1.0008 

1.1250 

1.0417 

1.0000 

1.2963 

1.0234 

1.4783 

3 

1.4359 

1.0052 

1.2122 

1.0102 

1.0085 

1.1766 

1.0036 

1.3047 

4 

1.5625 

1.0208 

1.2868 

1.0008 

1.0234 

1.1144 

1.0000 

1.2133 

5 

1.6718 

1.0403 

1.3509 

1.0007 

1.0390 

1.0775 

1.0019 

1.1577 

6 

1.7679 

1.0607 

1.4067 

1.0048 

1.0538 

1.0538 

1.0058 

1.1208 

7 

1.8533 

1.0811 

1.4560 

1.0111 

1.0675 

1.0378 

1.0105 

1.0947 

8 

1.9302 

1.1009 

1.5000 

1.0185 

1.0800 

1.0267 

1.0155 

1.07  57 

9 

2.0000 

1.1200 

1  5396 

1.0264 

1.0915 

1.0187 

1.0204 

1.0612 

10 

2.0639 

1.1382 

1.5756 

1.0345 

1 . 1020 

1.0130 

1.0252 

1.0500 

20 

2.5067 

1.2812 

1.8182 

1.1063 

1.1743 

1.0003 

1.0618 

1.0079 

30 

2.7724 

1.3772 

1.9582 

1.1581 

1.2158 

1.0066 

1.0846 

1.0006 

40 

2.9582 

1.4478 

2.0538 

1.1967 

1.2438 

1.0146 

1.1004 

1.0002 

50 

3.0990 

1.5029 

2.1250 

1.2269 

1.2645 

1.0220 

1.1122 

1.0017 

60 

3.2111 

1.5476 

2.1809 

1.2514 

1.2805 

1.0286 

1.1214 

1.0037 

70 

3.3034 

1.5850 

2.2265 

1.2718 

1.2935 

1.0344 

1.1289 

1.0059 

80 

3.3813 

1.6169 

2.2647 

1.2892 

1.3044 

1.0396 

1.1352 

1.0081 

90 

3.4483 

1.6446 

2.2973 

1.3044 

1.3136 

1.0441 

1.1405 

1.0102 

100 

3. 5069 

1.6690 

2.3257 

1.3176 

1.3215 

1.0482 

1.1451 

1.0121 
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5.  THE  EFFECTS  OF  VARIANCE  AND  AUTOCORRELATION 

It  is  instructive  to  examine  the  effect  of  population  variance 
and  autocorrelation  on  the  efficient  time  allocation.  Assuming  the 
two  samples  are  independent,  we  consider  two  cases  of  special  interest 


Case  l : 

2 

.  2 

*1 

4  o2 

Pl,s  ■  p2,s> 

Case  2: 

2 

al 

2 

*  a2 

Pl  ,s  4  p2  ,s ' 

For  Case  1  we  note  that 

(10)  r  »  o\/o\  * 

which  is  easily  estimated.  Figure  l  can  then  be  used  to  choose  an 
a.  One  might  therefore  conclude  that  if  the  nature  of  autocorrelation 
in  each  sample  is  roughly  alike,  an  efficient  choice  of  a  can  be 
based  on  the  ratio  of  the  population  variances.  This  conjecture  may 
be  contrary  to  fact  as  we  now  show  using  Case  2. 

For  convenience  of  exposition  we  assume 

(1  la)  pL  t  -  oj1!  0  fi  oL  <  1  1-1,2. 

This  autocorrelation  function  corresponds  to  the  Markov  process, 


Xl,t  *  «lXl.t-l  +  €l.t  * 


where  (e.  1  is  a  sequence  of  mutually  Independent,  Identically 

L  »  t 

distributed  random  variables.  Now  we  have 


-  [I  -  0^2  +  (otj-o-j)] /[l  -  0'1»2  '  (Oj-OTj)]  • 


(He) 


r 
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Figure  2  shows  r  as  a  function  of  for  several  values  of  or^ . 

Notice  that  for  a  given  difference,  (q^-q^)  .  r  increases  rapidly  as 
CTj  and  therefore  increase.  This  result  implies  that  small  differ¬ 
ences  in  the  difference  are  important  when  each  sequence  is 

highly  au tocorrelated .  Hence,  while  it  may  appear  that  two  sequences 
are  quite  similar  because  their  autocorrelation  functions  are,  these 
minor  differences  may  strongly  affect  the  efficient  allocation  of  time 
between  the  experiments. 

6.  CORRELATED  SAMPLES 

It  is  often  suggested  that  one  may  reduce  the  sample  sl2es  needed 
to  obtain  a  given  accuracy  V  by  working  with  positively  correlated 
samples  oi  the  two  experiments.  This  is  clearly  true  for  equal  sample 
sizes.  Here  the  effects  of  correlated  sampling  on  the  efficient 
choice  of  and  N2  are  investigated. 

The  expression  to  be  minimized,  in  analogy  to  (5c),  Is 

(12)  L  *  c1Nl -I- c2N2+  X[V-m, /NI-m2/N2+  2m;}/max(N1  ,N2)]  . 

This  yields 

( 1 3a)  »  {mj  +  [bm2(ml-2m3)"|^  -  2m /V 

( 1 3 b)  N*  =  fm2+  [m2(ml-2m3)  /b]^l/V 

for 

( l 3c)  m2  S  b(m^-2m3) 


and 
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(14a) 
( 14b) 


Nj  “  {mj  +  [bnij  (m2-2m3)]^}/V 

n2  '  f[m1(m2-2m3) /b]^  +  m2  -  2m3} /V 


for 


(14c) 


m2  S  bm^  +  2m3 . 


It  is  easily  shown  that  correlated  samples  reduce  the  required  computer 
time  when  either  (13c)  or  (14c)  holds. 

It  may  occur  that  m2  sat.^fies  neither  constraint  so  that 


(15) 


b(mj-2m3)  <  m2  <  bm^  +  2m3 . 


Suppose  we  then  choose  our  sample  sizes  as 


(16a) 


Nl  '  N2  “  (m1+m2"2m3) /V’ 


The  corresponding  computer  time  is 


(16b) 


T'  =  (cj+c2)  (m1'l-m2-2m3)  /V. 


As  an  alternative,  in  the  case  where  (13)  holds,  we  may  work  with 
independent  samples  and  choose  and  N2  as  in  (6a)  and  (6b).  The 


required  computer  time  is  then 
(17) 


T"  =  [(Cjmj)*  +  (c2m2)V/V. 


For  independent  sampling  to  be  desirable,  we  require 


(18) 


T'  -  T"  >  0. 


I.xpresslon  (18),  however,  implies  that 


>  bmj  +  2(l+b)m3  +  2[Zb( l+b)mjm3]  , 


(19a) 
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(19b)  bm^  >  m2  +  2(l+b)m3+2[2(l+b)m2m3]^ 

which  clearly  violate  (15).  When  (15)  holds  it  is  therefore  desirable 
to  work  with  correlated  samples  with  equal  sample  sizes  determined  by 
( 1 6a)  . 

In  practice  the  quantities  ,  n>2  and  m3  are  unknown  prior  to 
experimentation.  tj  estimate  them  an  initial  sample  of  say  Q  obser¬ 
vations  on  each  experiment  is  needed.  The  estimation  procedure  is 
described  in  Sec.  7.  For  now  we  assume  (  m2  and  m3  are  known  and 
we  concern  ourselves  with  several  contingencies  that  may  arise. 

Suppose  that  (13c)  or  (14c)  holds.  If 

(20)  N*  >  Q,  N*  Jt  Q, 

no  difficulty  occurs.  If,  however, 

(2la)  N*  >  Q,  N*  <  Q, 

•k  k 

then  we  recompute  and  N2  as 

(21b)  N*  -  Q(ml-2m3)/(QV-m2) 

(2lc)  N*  -  Q. 

Similarly  when 

(22a)  N*  <  Q,  N*  ?  Q, 

then  we  set 

(22b)  N*  «  Q 

N*  -  Q(m2-2m3) /(QV-m{) . 


(22c) 
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If  both  required  sample  sizes  are  less  thanQ,  we  terminate  the 
experiments . 

7.  ESTIMATION  PROCEDURE 

The  quantities  m^  and  m^  seldom  are  known  a  priori  and  there¬ 
fore  a  method  of  estimating  them  is  required.  The  presence  of  auto¬ 
correlation  in  t]  and  {X^  makes  estimation  somewhat  difficult. 
Ideally  we  desire  a  method  that  can  easily  be  incorporated  into  the 
performance  of  the  experiments.  Then  sample  sizes  can  be  automatically 
determined  within  the  simulations. 

The  method  suggested  here  accounts  for  autocorrelation  but  unfor¬ 
tunately  cannot  be  readily  incorporated  into  the  experiments.  It 
nevertheless  provides  valuable  assistance  to  the  experimenter  especially 
when  large  amounts  of  computer  ame  are  involved.  The  method  has  been 
described  in  [3].  Here  we  briefly  describe  and  extend  it  to  the  esti¬ 
mation  of  m^ .  Several  clarifying  remarks  on  [3]  should  facilitate  its 
application. 

Suppose  we  initially  collect  Q  observations  on  experiment  1  and 
wish  to  estimate  m^.  Since  the  same  procedure  applies  to  m2  in 
experiment  2,  we  momentarily  drop  the  subscript  i.  The  sample  auto¬ 
covariance  function  is 

Q-t 

(23a)  Rt  -  Q'1^  [(Xt-X)(Xt+T-X)] 

t  -  0,1  ....  ,M  <  Q 


(23b) 
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which  for  large  Q  has  expectation 


(23c) 


e(r  )  ~  a  o  *  m/Q. 

T  T 


An  initial  estimate  of  m  Is 


(24  a) 


n 

■  «o  + 


T“l 


which  for  appropriately  chosen  M  has  expectation 


(24  b) 


E(m)  ~  m(  1  -M/Q) 


Dividi  ig  (24a)  by  (l-M/Q)  compensates  for  the  bias  so  that  oar  esti¬ 
mator  of  m  is 


M 


(25a) 


=  r*0  +  (1-t/M)Rt] /(l-M/Q). 


T*l 


If  Is  a  normal  process  then 


(25  b) 


var(m)  ~  Ym 


(25c) 


Y  -  4M/(3Q)  . 


The  experimenter  must  choose  an  appropriate  value  for  the  design 
parameter  M.  Our  clarifying  remarks  concern  this  choice.  Notice  that 
as  M  becomes  large 

M 

(26)  lim  7  (1-|t|/M)R  -m  R  -  02p  , 

the  desired  mathematical  limit.  In  (24a)  we  use  R  since  R  is  unknown. 

T  T 

2 

As  M  increases  here  the  variance  Ym  also  increases  so  that  the  esti¬ 
mate  becomes  less  statistically  reliable.  IVtls  dilemma  between  good 
resolution  (mathematical  convergence)  and  good  reliability  (convergence 
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in  probability)  is  fundamental  to  spectrum  analysis.  Clearly  a 
compromise  must  be  made  which  permits  both  acceptable  resolution  and 
reliabil ity . 

The  compromise  depends  on  the  choice  of  M.  To  guide  us  in  our 
choice  we  use  the  ratio 

(27a)  p  «  [var(m)]^/E(m)  ~  [4M/(3Q)]V 

If 

(27  b)  M  -  3Q/4 

the  ratio  p  is  unity  which  implies  that  sampling  fluctuations  in  m 
are  roughly  of  the  same  order  as  its  mean  m.  This  is  clearly  undesir¬ 
able.  If 

(27c)  M  -  Q/4 

then  the  ratio  is  roughly  .57.  It  is  suggested  that  M  not  exceed  Q/4 
and  in  general  be  kept  much  smaller.  One  may  easily  compute  m  for 
several  values  of  M,  say  Q/32,  Q/16,  Q/8,  3Q/16  and  Q/4.  Doing  so 
permits  the  experimenter  to  decide,  albeit  subjectively,  when  m  is 
we 1 1  resolved . 

There  is  the  possibility  that  Q  may  be  too  small  to  permit  good 
resolution.  If  the  estimate  m  fails  to  become  well  resolved  as  Q 
increases  to  M/4,  then  more  observations  should  be  collected  and  used. 
Failure  to  obtain  a  well-resolved  estimate  can  lead  to  a  serious 
underestimate  of  m. 

The  estimation  of  m^  Is  accomplished  In  the  same  manner.  Suppose 
we  have  Q  observations  on  each  experiment.  We  form  the  sample  covar¬ 


iance  function 
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Q*  i  t| 

512.T">''E(,‘l,t-Xl)<X2,[+T-X2) 


T  *  0  ,il , . . .  ,±M, 


which  for  large  Q  has  expectation 


E(R12,t)  ~  R12,t  *m3/Q- 


Our  estimator  Is  then 


( l  -  )  r  j  /M)  R.  5  ,  /(1-M/Q) 


with  expectation 


E(m^)  ~  m3 


and  variance,  when  both  processes  are  normal  [6], 


var(nl3)  ~  f  /2. 


In  spectrum  analysis  the  quantity  m3/rr  Is  called  the  co-spectrum 
of  fx^  t)  and  {X^  at  frequency  sero.  The  same  procedure  concerning 
Q  applies  here  as  In  the  estimation  of  m^  and  m^,. 

★  -k 

Our  maLn  purpose  Is  to  estimate  Nj  and  .  When  (13c)  holds,  we 
have  Initial  estimates 


Nt  -  faL  +  [l*fi2(fll-2i63)]if-2ffi3}/V 
N*  -  f^2  +  C*2<ai -2a3)  /V. 


When  (14c)  holds,  the  Initial  estimates  are 


(30a) 
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(30b)  ^  -  {m2  +  [m1(m2-2m  )/b]^  -2m3}/V. 

When  (15)  holds,  the  sample  sizes  are  equal  so  that 
(31)  ^  =  (fii1+m2-2n»3)  /V  . 

If  and  { X2  are  normal  processes  we  may  use  the  results 

In  [6]  to  derive  the  sampling  properties  of  1?^  and  ^  to  order  Q*1. 
Defining 

(32a)  2^  s  mi-2m3 

(32b)  9t  s  (m3-m1m2)(m1+m2-2m3)/(4V2m3i0^  )^  1  ■  1,2 

lb  1=1 

(32c)  6  = 

(l/b  1-2 


and  noting  that 

(32d) 

E(3t)  —  0 1  s  mt-2m3 , 

we  may  show  that 

(32e) 

cov(mi>m3)  ~  Ym^ir^ 

(  32  f ) 

2  2 
varC^)  ~  +  2(m1m2+m3)  ] 

(32g) 

2 

cov(2l,mj)  ~  Y(m3-2mJm3)  i,j  -  1,2;  l*j 

When  (13c)  holds  we  have 

e(H*-n*)  ~  'fel6l 


(33a) 
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(33b)  V2var(tl^)  ~  [  l  +■  (6^/0^  ^]2  var(5>1)  +  2[  (5  ^  +  6j]  cov^.n^) 

+  var(m2) 

(33c)  V2var(?f^)  ~  ^ ^2m2/ '  1^  var(-i*  +  2[(&2m2^1^  +  52^  cov^i’V 

\  2 

+  II  +  (  2®l^m2^  ^  var(m2) . 

When  (14c)  holds  we  have 

(34a)  E(S^-N*)  ~  Ve25i 

(34b)  V2var(I^)  ~  ( 6 /c>2)  var(32)  +  2[  (S^/^)  ^  +  6^]  cov^.m^) 

+  (1  +  ( 6 ^  2  var(™i) 

(34c)  V2varOJ*)  ~  [1  +  (62ml/?2^2  var (32>  +  2[  ( i^/n^) ^  +  62]  cov^.rn^ 
+  (S^/nij)  var(rh^)  . 

Since 

2 

(35a)  m3  < 

(35b)  +  m2  >  2mj , 

the  bias  Is  negative.  To  compensate  for  this  bias  we  estimate  N*  by 

(35c)  "I-^Vi 

when  (13c)  holds,  and  by 

(35d)  N*  -  ^-yi26  t 

when  (14c)  holds.  Here  we  define 

et  a  (<h3-fh1m2)(ml4A2-2m3)/(4V2iSl3_l^J)^ 


(35e) 


1-1,2. 


-22- 


To  order  Q  the  variances  remain  unchanged. 

When  (IS)  holds  equal  sample  sizes  are  used  and  we  have  straight¬ 
forwardly 

(36a)  E(SfJ)  ~  N* 

(36b)  var(U^)  ~  YlKm^+m^)^  +  ^my'in^-m^-n^)]  . 

No  bias  adjustment  is  necessary  here. 

For  normal  processes  the  distributions  if  m^/m^  and  m^/m^  are 

usually  approximated  by  that  of  chi-square.  This  suggests  that  we 

*  ^ 

also  approximate  the  distribution  of  N^/N^  by  that  of  chi-square  with 
degrees  of  freedom 

(37)  fL  »  2[E(N*)]2/var(N*). 

Estimating  f^  by  replacing  the  terms  in  (37)  by  their  corresponding 

sample  values  will  assist  the  experimenter  in  determining  how  reliable 

'*  A* 

his  estimate  of  N^  is.  This  is  especially  important  when  c^N^  turns 

out  to  be  large. 

8.  ANTITHETIC  VARIATES 


It  may  occur  that  the  sample  sizes  determined  by  the  foregoing 
analysis  require  a  prohibitively  large  amount  of  computer  time.  In 
this  case  positively  correlated  samples  have  not  sufficed  to  bring  the 
cost  of  experimentation  within  reasonable  bounds.  One  may  Improve  this 
situation  by  working  with  negatively  correlated  replications  of  each 
experiment.  The  suggested  technique  is  called  the  method  of  antithetic 
variates  ,5],  If  is  the  i  uniformly  distributed  psuedorandom  number 


in  the  first  replication  of  the  experiment,  then  using  (1  -  §t)  as  the 
uniformly  distributed  pseudorandom  number  in  the  corresponding  event 
of  the  antithetic  replication  induces  a  negative  correlation  between 
the  observations  in  the  two  replications. 

th  ^ 

Let  X  be  an  observation  on  the  j 1  replication  of  the  i 
i  J  •  t 

experiment  at  time  t.  Suppose  we  collect  observations  on  each 

replication  of  experiment  l.  Then  we  have 

N 

i 

(38a)  *i)  =  *i  £  *ij,t  j*1’2’ 

t=l 


(38b)  Xt  =  (Xa  +  Xi2)/2. 

If  the  second  replication  uses  the  antithetic  pseudorandom  number 
sequence  of  the  first,  then 

(38c)  cov(Xu,  Xl2)  =  4/**!. 

mi,4  ^  ° 

so  that 

( 38d)  var(X.)  -  (m^  4)/(2ty. 

If  nii  ^  *  m^/2  then  using  two  antithetic  replications  of  length  is 
equivalent  to  collecting  4N^  observations  on  l  replication.  If 
m^  ^  -  3m^/4  then  two  antithetic  replications  each  of  length  is 
equivalent  to  8N^  observations  on  one  replication. 

To  incorporate  antithetic  variates  into  the  preceding  results, 


we  need  only  form  the  observations  defined  by  (38b)  and  compute  m^ , 


-24- 


•ff  "ff 

m2  and  as  described.  The  resulting  means  that  N^/2  observations 
should  be  collected  on  each  of  two  antithetic  replications  of  experl- 
ment  1.  A  similar  result  applies  to  Nj . 

9 .  CONCLUSIONS 

Our  results  indicate  that  for  a  given  level  of  accuracy,  signifi¬ 
cantly  less  computer  time  is  required  when  the  sample  sizes  are  deter¬ 
mined  according  to  the  derived  rule  than  when  the  sample  sizes  are  equal. 
This  is  true  when  one  uses  either  independent  or  positively  correlated 
samples  on  the  two  experiments. 

An  interesting  conclusion  also  emerges  about  the  significance  of 
the  degree  of  autocorrelation  in  the  observed  processes.  Our  results 
imply  that  small  differences  in  the  autocorrelation  functions  are  im¬ 
portant  for  the  time  allocation  when  each  process  is  highly  autocorrelated . 
Failure  to  take  these  differences  into  account  can  significantly  affect 
the  efficient  allocation  of  time,  thereby  requiring  unusually  long  running 
times  to  obtain  a  given  level  of  precision. 

The  suggested  two-stage  procedure  provides  initial  estimates  for 
determining  sample  sizes  and  final  estimates  for  testing  hypotheses. 

The  graphical  analysis  suggests  that  the  efficient  allocation  is  not 
very  sensitive  to  small  errors  in  the  estimates  and,  therefore,  the 
use  of  estimates  for  the  population  parameters  is  not  of  major  concern. 

It  is  important,  however,  to  get  good  estimates  of  m^ ,  and  m^.  This 

depends  on  an  appropriate  choice  of  M  using  the  suggested  guidelines. 
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